Skip to content

Comments

fix(audit): surface pre-agent step errors when agent never executed#17623

Merged
pelikhan merged 2 commits intomainfrom
copilot/fix-audit-error-summary
Feb 22, 2026
Merged

fix(audit): surface pre-agent step errors when agent never executed#17623
pelikhan merged 2 commits intomainfrom
copilot/fix-audit-error-summary

Conversation

Copy link
Contributor

Copilot AI commented Feb 22, 2026

When a workflow fails before the agent runs (lockdown validation, missing secrets, binary install failures), audit reports "No specific errors identified" despite the actual error being present in downloaded step log files. The errors array was always empty since agent-stdio.log-based extraction was removed, leaving no fallback.

Changes

pkg/cli/audit_report.go

  • extractPreAgentStepErrors(): when agent-stdio.log is absent and workflow conclusion is failure, scans workflow-logs/{job}/{step_num}_{step_name}.txt and returns the last executed step's content (highest step number = likely failing step)
  • parseStepFilename(): parses 12_Validate lockdown mode requirements.txt(12, "Validate lockdown mode requirements")
  • stripGHALogTimestamps(): strips 2024-01-01T10:00:00[.sss...]Z prefixes from log lines; handles variable-length fractional seconds
  • buildAuditData(): calls extractPreAgentStepErrors for failed runs, feeding the result into generateFailureAnalysis so error_summary reflects the actual failure

pkg/cli/audit.go

  • fetchWorkflowRunMetadata(): returns a human-readable error for non-existent run IDs instead of propagating the raw exit status 1

Before / After

// Before
"failure_analysis": {
  "primary_failure": "failure",
  "failed_jobs": ["agent"],
  "error_summary": "No specific errors identified"
}

// After
"failure_analysis": {
  "primary_failure": "failure",
  "failed_jobs": ["agent"],
  "error_summary": "Lockdown mode is enabled (lockdown: true) but no custom GitHub token is configured.\n\nPlease configure one of the following as a repository secret:\n  - GH_AW_GITHUB_TOKEN (recommended)\n  ...",
  "root_cause": ""
}

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/graphql
    • Triggering command: /usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw GO111MODULE 64/bin/go infocmp -1 xterm-color go 3313510/b430/workflow.test -json GO111MODULE 64/bin/go 3313510/b430/workflow.test (http block)
  • https://api.github.com/repos/actions/ai-inference/git/ref/tags/v1
    • Triggering command: /usr/bin/gh gh api /repos/actions/ai-inference/git/ref/tags/v1 --jq .object.sha jpq_JbW5I GO111MODULE /opt/hostedtoolcache/go/1.25.0/x64/bin/go l GOMOD GOMODCACHE ortcfg env g/timeutil/format.go g/timeutil/format_test.go /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/vet (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v3
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v3 --jq .object.sha 7848/001/stability-test.md GO111MODULE ache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE go env 1520-27702/test-1652831907/.github/workflows GO111MODULE .cfg GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v4 --jq .object.sha 01 GO111MODULE /opt/hostedtoolcache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE go env runs/20260222-001520-27702/test-3457897160/.github/workflows GO111MODULE /opt/hostedtoolcache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v4 --jq .object.sha -json rsion=abf14b3-dirty /opt/hostedtoolcache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE .cfg GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v4 --jq .object.sha uts.version Test User /usr/bin/git -json GO111MODULE 64/bin/go git rev-�� --show-toplevel node /opt/hostedtoolcache/node/24.13.0/x64/bin/node --check **/*.cjs 64/bin/go node (http block)
  • https://api.github.com/repos/actions/checkout/git/ref/tags/v5
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE ache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha ithub/workflows/bot-detection.md GOPROXY /usr/bin/git GOSUMDB GOWORK 64/bin/go git rev-�� s/test.md go /usr/bin/git -json GO111MODULE 64/bin/go git (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha user.name Test User /usr/bin/git che/go-build/d7/git GOPROXY 64/bin/go git rev-�� --show-toplevel /opt/hostedtoolcache/go/1.25.0/xGO111MODULE /usr/bin/gh /tmp/go-build248git -trimpath 64/bin/go gh (http block)
  • https://api.github.com/repos/actions/github-script/git/ref/tags/v8
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha GOSUMDB GOWORK 64/bin/go GOINSECURE GOMOD GOMODCACHE go env ck 'scripts/**/*GOINSECURE GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha go1.25.0 -c=4 -nolocalimports -importcfg /tmp/go-build2203313510/b389/importcfg -pack /tmp/go-build2203313510/b389/_testmain.go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD erignore go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/actions/setup-go/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v4 --jq .object.sha i/install.sh GO111MODULE /opt/hostedtoolcache/go/1.25.0/x-nilfunc GOINSECURE GOMOD GOMODCACHE go env 1520-27702/test-1895399 GO111MODULE /opt/hostedtoolcache/go/1.25.0/x64/bin/go l GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/actions/setup-node/git/ref/tags/v4
    • Triggering command: /usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq .object.sha /ref/tags/v8 GO111MODULE Name,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE /opt/hostedtoolcache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/1/artifacts
    • Triggering command: /usr/bin/gh gh run download 1 --dir test-logs/run-1 GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh run download 1 --dir test-logs/run-1 GO111MODULE ode_modules/.bin/node GOINSECURE GOMOD GOMODCACHE go ode_�� -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12345/artifacts
    • Triggering command: /usr/bin/gh gh run download 12345 --dir test-logs/run-12345 GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go estl�� ty-test.md GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh run download 12345 --dir test-logs/run-12345 GO111MODULE _modules/.bin/node GOINSECURE GOMOD GOMODCACHE go ode_�� -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/12346/artifacts
    • Triggering command: /usr/bin/gh gh run download 12346 --dir test-logs/run-12346 GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE debian-sa1 estl�� 1 GOPROXY 64/bin/go GOSUMDB GOWORK modules/@npmcli/add go (http block)
    • Triggering command: /usr/bin/gh gh run download 12346 --dir test-logs/run-12346 GO111MODULE es/.bin/node GOINSECURE GOMOD GOMODCACHE go ode_�� -json GO111MODULE 64/bin/go N files are not git GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/2/artifacts
    • Triggering command: /usr/bin/gh gh run download 2 --dir test-logs/run-2 GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env ty-test.md GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh run download 2 --dir test-logs/run-2 GO111MODULE odules/npm/node_modules/@npmcli/run-script/lib/node-gyp-bin/node GOINSECURE GOMOD GOMODCACHE go ode_�� -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/3/artifacts
    • Triggering command: /usr/bin/gh gh run download 3 --dir test-logs/run-3 GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh run download 3 --dir test-logs/run-3 GO111MODULE _modules/.bin/node GOINSECURE GOMOD GOMODCACHE go ode_�� -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/4/artifacts
    • Triggering command: /usr/bin/gh gh run download 4 --dir test-logs/run-4 GO111MODULE x_amd64/compile GOINSECURE GOMOD GOMODCACHE x_amd64/compile env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh run download 4 --dir test-logs/run-4 GO111MODULE es/.bin/node GOINSECURE GOMOD GOMODCACHE go ode_�� -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/runs/5/artifacts
    • Triggering command: /usr/bin/gh gh run download 5 --dir test-logs/run-5 GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh run download 5 --dir test-logs/run-5 GO111MODULE node GOINSECURE GOMOD GOMODCACHE go ode_�� -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path GOSUMDB GOWORK 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 100 GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE node (http block)
    • Triggering command: /usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 6 GOMOD GOMODCACHE x_amd64/vet env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.0.0
    • Triggering command: /usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.0.0 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env 3603486071/.github/workflows GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/nonexistent/action/git/ref/tags/v999.999.999
    • Triggering command: /usr/bin/gh gh api /repos/nonexistent/action/git/ref/tags/v999.999.999 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/nonexistent/repo/actions/runs/12345
    • Triggering command: /usr/bin/gh gh run view 12345 --repo nonexistent/repo --json status,conclusion GOINSECURE GOMOD GOMODCACHE x_amd64/vet env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh run view 12345 --repo nonexistent/repo --json status,conclusion GOINSECURE GOMOD GOMODCACHE go env th .prettierignore GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
  • https://api.github.com/repos/owner/repo/actions/workflows
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo 64/bin/go GOINSECURE GOMOD GOMODCACHE go env tformat GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh workflow list --json name,state,path --repo owner/repo 64/bin/go /home/REDACTED/worgo go /usr/bin/git sh -c "prettier" --wriGOINSECURE git 64/bin/go --show-toplevel x_amd64/compile /usr/bin/git go (http block)
  • https://api.github.com/repos/owner/repo/contents/file.md
    • Triggering command: /tmp/go-build2203313510/b380/cli.test /tmp/go-build2203313510/b380/cli.test -test.testlogfile=/tmp/go-build2203313510/b380/testlog.txt -test.paniconexit0 -test.v=true -test.parallel=4 -test.timeout=10m0s -test.run=^Test -test.short=true GOINSECURE GOMOD ode-gyp-bin/sh go k/gh�� -json GO111MODULE 64/bin/go GOINSECURE GOMOD erignore go (http block)
    • Triggering command: /tmp/go-build290692283/b001/cli.test /tmp/go-build290692283/b001/cli.test -test.testlogfile=/tmp/go-build290692283/b001/testlog.txt -test.paniconexit0 -test.timeout=5m0s -test.run=^Test -test.short=true security 64/bin/go OUTPUT -d 168.63.129.16 node /hom�� --write **/*.cjs 64/bin/go **/*.json --ignore-path run-script/lib/n"prettier" --check '**/*.cjs' '**/*.ts' '**/*.json' --ignore-path ../../../.prettierignore sh (http block)
  • https://api.github.com/repos/test-owner/test-repo/actions/secrets
    • Triggering command: /usr/bin/gh gh api /repos/test-owner/test-repo/actions/secrets --jq .secrets[].name GOSUMDB GOWORK 64/bin/go GOINSECURE GOMOD GOMODCACHE go m/_n�� -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go (http block)
    • Triggering command: /usr/bin/gh gh api /repos/test-owner/test-repo/actions/secrets --jq .secrets[].name --noprofile git 64/bin/go --get remote.origin.urenv /usr/bin/git sh -c "prettier" --wriGOINSECURE git 64/bin/go /ref/tags/v8 go /usr/bin/git make (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Original prompt

This section details on the original issue you should resolve

<issue_title>[cli-tools-test] audit: "No specific errors identified" for failed runs when agent never executed</issue_title>
<issue_description>## Problem Description

When a workflow run fails in a pre-agent step (before the AI agent executes), the audit tool reports "No specific errors identified" in the failure_analysis.error_summary field, despite the specific error being present in the downloaded step log files.

Steps to Reproduce

  1. Trigger a workflow run that fails before the agent step (e.g., lockdown mode validation failure)
  2. Audit the failed run: gh aw audit (run_id) (or via MCP audit tool)
  3. Observe the failure_analysis.error_summary in the report

Example run ID: 22266577612 (Issue Monster, failed 2026-02-21T23:40:03Z)

Expected Behavior

The audit report should extract and surface the specific error message from the step log files. For example, from workflow-logs/agent/12_Validate lockdown mode requirements.txt:

Lockdown mode is enabled (lockdown: true) but no custom GitHub token is configured.

Please configure one of the following as a repository secret:
  - GH_AW_GITHUB_TOKEN (recommended)
  - GH_AW_GITHUB_MCP_SERVER_TOKEN (alternative)
  - Custom github-token in your workflow frontmatter

Actual Behavior

The audit report shows:

{
  "failure_analysis": {
    "primary_failure": "failure",
    "failed_jobs": ["agent"],
    "error_summary": "No specific errors identified"
  }
}
  • No agent-stdio.log is downloaded (agent never ran)
  • run_summary.json shows "errors": null
  • The actual error exists in downloaded step log files but is not surfaced

Root Cause

The audit tool appears to extract errors primarily from agent-stdio.log (the AI agent's output). When the agent job fails at an early step (e.g., step 12 out of 67+), there's no agent output. The tool doesn't fall back to scanning the step-level log files in workflow-logs/agent/ to find the failure message.

Environment

  • Repository: github/gh-aw
  • Run ID (testing session): 22266663143
  • Failed run audited: 22266577612
  • Date: 2026-02-21
  • gh-aw version: 0.0.414 (agent_version from aw_info.json)

Impact

  • Severity: High
  • Frequency: Always (any workflow that fails before agent execution)
  • Workaround: Manually inspect the step logs in workflow-logs/agent/ directory
  • Affected users: Anyone using audit to debug failed workflows, where the failure is a configuration issue (missing secrets, lockdown validation, etc.)

Additional Context

Common scenarios where the agent never executes (and thus this bug manifests):

  • Missing required secrets in lockdown mode
  • Failed MCP gateway setup
  • Failed binary installation (awf, copilot CLI)
  • Failed repository checkout

The audit is most valuable precisely in these failure scenarios, making this a significant gap in debugging capability.

Related: The audit command also returns a cryptic exit status 1 error when given an invalid run ID (e.g., 99999999999), instead of a human-readable "Run not found" message. Both issues reduce the audit tool's usefulness for diagnosing problems.

Generated by Daily CLI Tools Exploratory Tester

  • expires on Feb 28, 2026, 11:55 PM UTC

Comments on the Issue (you are @copilot in this section)


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

…ent never ran

When a workflow run fails in a pre-agent step (e.g., lockdown validation,
missing secrets), the audit tool now extracts the actual error message from
the step log files in workflow-logs/{job}/{step_num}_{step_name}.txt instead
of reporting "No specific errors identified".

Also improves the error message for invalid run IDs to provide a human-readable
message instead of the cryptic "exit status 1".

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix audit tool to display specific errors for failed runs fix(audit): surface pre-agent step errors when agent never executed Feb 22, 2026
@pelikhan pelikhan marked this pull request as ready for review February 22, 2026 03:52
Copilot AI review requested due to automatic review settings February 22, 2026 03:52
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes a significant gap in the audit command where failed workflow runs that never executed the agent step reported "No specific errors identified" despite the actual error being present in downloaded step log files. The fix extracts and surfaces the actual failure messages from GitHub Actions step logs when the agent never ran.

Changes:

  • Adds extraction of pre-agent step errors from workflow-logs when agent-stdio.log is absent and workflow failed
  • Implements timestamp stripping for GitHub Actions log format to provide clean error messages
  • Improves error messaging for non-existent run IDs with actionable guidance

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
pkg/cli/audit_report.go Implements extractPreAgentStepErrors to scan step logs, parseStepFilename to parse step numbering, stripGHALogTimestamps to clean log output, and integrates error extraction into buildAuditData
pkg/cli/audit_report_test.go Comprehensive test coverage for all new functions including edge cases, timestamp formats, truncation, and integration with buildAuditData
pkg/cli/audit.go Adds human-readable error message when workflow run ID is not found instead of generic "exit status 1"

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +540 to +569
var lastStep *stepLog

jobDirs, err := os.ReadDir(workflowLogsDir)
if err != nil {
return nil
}

for _, jobEntry := range jobDirs {
if !jobEntry.IsDir() {
continue
}
jobDir := filepath.Join(workflowLogsDir, jobEntry.Name())
stepFiles, err := os.ReadDir(jobDir)
if err != nil {
continue
}
for _, stepFile := range stepFiles {
if stepFile.IsDir() || !strings.HasSuffix(stepFile.Name(), ".txt") {
continue
}
num, stepName := parseStepFilename(stepFile.Name())
if num > 0 && (lastStep == nil || num > lastStep.num) {
lastStep = &stepLog{
path: filepath.Join(jobDir, stepFile.Name()),
num: num,
stepKey: jobEntry.Name() + "/" + stepName,
}
}
}
}
Copy link

Copilot AI Feb 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current logic finds the step with the highest step number across ALL jobs in workflow-logs. In a multi-job workflow where a job fails early (e.g., "setup" fails at step 5) but another job successfully runs later steps (e.g., "build" completes steps 1-15), this would extract content from the "build" job instead of the failing "setup" job.

While this works correctly for single-job workflows (which appears to be the primary use case based on the issue description mentioning "the agent job"), it may produce misleading results in multi-job scenarios.

Consider documenting this limitation in the function comment, or as a future enhancement, filtering to only examine jobs that failed (though this would require passing job information to this function).

Copilot uses AI. Check for mistakes.
@pelikhan pelikhan merged commit 964cbd7 into main Feb 22, 2026
141 checks passed
@pelikhan pelikhan deleted the copilot/fix-audit-error-summary branch February 22, 2026 03:57
github-actions bot added a commit that referenced this pull request Feb 22, 2026
- Document audit command's improved error surfacing for pre-agent
  failures (#17623)
- Document automatic parent author mention preservation in add-comment
  (#17628)
- Document fail-fast behavior when code push safe outputs fail (#17543)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[cli-tools-test] audit: "No specific errors identified" for failed runs when agent never executed

2 participants